分层多粒度分类（HMC）将分层多粒度标签分配给每个对象，专注于对标签层次结构进行编码，例如[“Albatross”，“Laysan Albatross”]从粗略级别进行。然而，细粒度的定义是主观的，并且图像质量可能会影响识别。因此，可以在层次结构的任何水平处观察样本，例如，例如，[“信天翁”]或[“白金贸易”，“Laysan Albatross”，并且在致动类别中辨别的示例在HMC的传统设置中通常被忽略。在本文中，我们研究了HMC问题，其中对象在层次结构的任何级别上标记。所提出的方法的基本设计源自两个动机：（1）学习在各个级别标记的物体应该转移级别之间的分层知识; （2）较低级别的类应继承与上级超类相关的属性。所提出的组合损失通过从树层次结构中定义的相关标签聚合信息来最大化观察到的地面真实标签的边际概率。如果观察到的标签处于叶片水平，则组合损失进一步施加了多级跨熵损失，以增加细粒度分类损失的重量。考虑到分层特征交互，我们提出了一个分层剩余网络（HRN），其中来自父级的粒度特定特征作为残留连接的特定特征被添加到儿童级别的特征。与最先进的HMC方法和精细的视觉分类（FGVC）方法相比，三种常用数据集的实验证明了我们的方法的有效性和利用标签层次结构的方法。

分子财产预测是药物和材料行业的基本任务。从物理上讲，分子的特性取决于其自身的电子结构，可以通过schr \“ odinger方程来精确描述。但是，由于大多数分子的求解schr \“ odinger”方程非常具有挑战性量子多体系统的行为。虽然已证明深度学习方法在分子性质预测中有效，但我们设计了一种新颖的方法，即GEM-2，它全面考虑了分子中的远距离和多体相互作用。 GEM-2由两个相互作用的轨道组成：一个原子级轨道模拟任意两个原子之间的局部和全局相关性，以及一个对所有原子对之间的相关性建模的成对轨道，它们嵌入任何3或4个原子之间的信息。广泛的实验证明了GEM-2在量子化学和药物发现任务中的多种基线方法的优越性。

基于AI的蛋白质结构预测管道（例如AlphaFold2）已达到了几乎实验的准确性。这些高级管道主要依赖于多个序列比对（MSA）和模板作为输入来从同源序列中学习共进化信息。但是，从蛋白质数据库中搜索MSA和模板很耗时，通常需要数十分钟。因此，我们尝试通过仅使用蛋白质的主要序列来探索快速蛋白质结构预测的极限。提出了Helixfold单一的形式将大规模蛋白质语言模型与AlphaFold2的优质几何学习能力相结合。我们提出的方法，Helixfold单个，首先预先培训是一种大规模蛋白质语言模型（PLM），使用了数以千计的主要序列利用自我监督的学习范式，将用作MSA和模板的替代方法共同进化信息。然后，通过将预训练的PLM和AlphaFold2的必需组件组合在一起，我们获得了一个端到端可区分模型，以仅从主要序列预测原子的3D坐标。 Helixfold-Single在数据集CASP14和Cameo中得到了验证，通过基于MSA的方法，具有大型同源家庭的基于MSA的方法，从而实现了竞争精度。此外，与主流管道进行蛋白质结构预测相比，Helixfold单个的时间比主流管道的时间少得多，这表明其在需要许多预测的任务中的潜力。 HelixFold-Single的守则可在https://github.com/paddlepaddle/paddlehelix/tree/dev/dev/pprotein_folding/helixfold-single上获得，我们还在https://paddlehelix.baidu.com上提供稳定的Web服务。 /app/drug/protein-single/prevast。

端到端的TTS需要大量的语音/文本配对数据，以涵盖所有必要的知识，尤其是如何在不同环境中发音不同的单词，以便神经模型可以相应地学习此类知识。但是，在实际应用中，很难满足对培训数据的高需求，并且通常需要手动注入其他知识。例如，为了捕获有关不正常拼字法的语言知识，需要基于大型结构化发音词典来构建复杂的字体到音调管道，从而导致额外的，有时很高的成本将神经TT扩展到此类语言。在本文中，我们提出了一个框架，以学习使用新颖的Token2知识注意模块从非结构化的外部资源中自动提取知识。该框架用于构建一个名为Neural Lexicon Reader的TTS模型，该模型以端到端的方式从Raw Lexicon文本中提取发音。实验表明，所提出的模型可显着减少低资源，端到端中国TT中的发音误差，并且可以将词典阅读能力转移到其他语言中，并具有较小的数据。

Deep learning models can achieve high accuracy when trained on large amounts of labeled data. However, real-world scenarios often involve several challenges: Training data may become available in installments, may originate from multiple different domains, and may not contain labels for training. Certain settings, for instance medical applications, often involve further restrictions that prohibit retention of previously seen data due to privacy regulations. In this work, to address such challenges, we study unsupervised segmentation in continual learning scenarios that involve domain shift. To that end, we introduce GarDA (Generative Appearance Replay for continual Domain Adaptation), a generative-replay based approach that can adapt a segmentation model sequentially to new domains with unlabeled data. In contrast to single-step unsupervised domain adaptation (UDA), continual adaptation to a sequence of domains enables leveraging and consolidation of information from multiple domains. Unlike previous approaches in incremental UDA, our method does not require access to previously seen data, making it applicable in many practical scenarios. We evaluate GarDA on two datasets with different organs and modalities, where it substantially outperforms existing techniques.

The development of social media user stance detection and bot detection methods rely heavily on large-scale and high-quality benchmarks. However, in addition to low annotation quality, existing benchmarks generally have incomplete user relationships, suppressing graph-based account detection research. To address these issues, we propose a Multi-Relational Graph-Based Twitter Account Detection Benchmark (MGTAB), the first standardized graph-based benchmark for account detection. To our knowledge, MGTAB was built based on the largest original data in the field, with over 1.55 million users and 130 million tweets. MGTAB contains 10,199 expert-annotated users and 7 types of relationships, ensuring high-quality annotation and diversified relations. In MGTAB, we extracted the 20 user property features with the greatest information gain and user tweet features as the user features. In addition, we performed a thorough evaluation of MGTAB and other public datasets. Our experiments found that graph-based approaches are generally more effective than feature-based approaches and perform better when introducing multiple relations. By analyzing experiment results, we identify effective approaches for account detection and provide potential future research directions in this field. Our benchmark and standardized evaluation procedures are freely available at: https://github.com/GraphDetec/MGTAB.

As one of the prevalent methods to achieve automation systems, Imitation Learning (IL) presents a promising performance in a wide range of domains. However, despite the considerable improvement in policy performance, the corresponding research on the explainability of IL models is still limited. Inspired by the recent approaches in explainable artificial intelligence methods, we proposed a model-agnostic explaining framework for IL models called R2RISE. R2RISE aims to explain the overall policy performance with respect to the frames in demonstrations. It iteratively retrains the black-box IL model from the randomized masked demonstrations and uses the conventional evaluation outcome environment returns as the coefficient to build an importance map. We also conducted experiments to investigate three major questions concerning frames' importance equality, the effectiveness of the importance map, and connections between importance maps from different IL models. The result shows that R2RISE successfully distinguishes important frames from the demonstrations.

Compressed videos often exhibit visually annoying artifacts, known as Perceivable Encoding Artifacts (PEAs), which dramatically degrade video visual quality. Subjective and objective measures capable of identifying and quantifying various types of PEAs are critical in improving visual quality. In this paper, we investigate the influence of four spatial PEAs (i.e. blurring, blocking, bleeding, and ringing) and two temporal PEAs (i.e. flickering and floating) on video quality. For spatial artifacts, we propose a visual saliency model with a low computational cost and higher consistency with human visual perception. In terms of temporal artifacts, self-attention based TimeSFormer is improved to detect temporal artifacts. Based on the six types of PEAs, a quality metric called Saliency-Aware Spatio-Temporal Artifacts Measurement (SSTAM) is proposed. Experimental results demonstrate that the proposed method outperforms state-of-the-art metrics. We believe that SSTAM will be beneficial for optimizing video coding techniques.

We propose a distributionally robust return-risk model for Markov decision processes (MDPs) under risk and reward ambiguity. The proposed model optimizes the weighted average of mean and percentile performances, and it covers the distributionally robust MDPs and the distributionally robust chance-constrained MDPs (both under reward ambiguity) as special cases. By considering that the unknown reward distribution lies in a Wasserstein ambiguity set, we derive the tractable reformulation for our model. In particular, we show that that the return-risk model can also account for risk from uncertain transition kernel when one only seeks deterministic policies, and that a distributionally robust MDP under the percentile criterion can be reformulated as its nominal counterpart at an adjusted risk level. A scalable first-order algorithm is designed to solve large-scale problems, and we demonstrate the advantages of our proposed model and algorithm through numerical experiments.

Witnessing the impressive achievements of pre-training techniques on large-scale data in the field of computer vision and natural language processing, we wonder whether this idea could be adapted in a grab-and-go spirit, and mitigate the sample inefficiency problem for visuomotor driving. Given the highly dynamic and variant nature of the input, the visuomotor driving task inherently lacks view and translation invariance, and the visual input contains massive irrelevant information for decision making, resulting in predominant pre-training approaches from general vision less suitable for the autonomous driving task. To this end, we propose PPGeo (Policy Pre-training via Geometric modeling), an intuitive and straightforward fully self-supervised framework curated for the policy pretraining in visuomotor driving. We aim at learning policy representations as a powerful abstraction by modeling 3D geometric scenes on large-scale unlabeled and uncalibrated YouTube driving videos. The proposed PPGeo is performed in two stages to support effective self-supervised training. In the first stage, the geometric modeling framework generates pose and depth predictions simultaneously, with two consecutive frames as input. In the second stage, the visual encoder learns driving policy representation by predicting the future ego-motion and optimizing with the photometric error based on current visual observation only. As such, the pre-trained visual encoder is equipped with rich driving policy related representations and thereby competent for multiple visuomotor driving tasks. Extensive experiments covering a wide span of challenging scenarios have demonstrated the superiority of our proposed approach, where improvements range from 2% to even over 100% with very limited data. Code and models will be available at https://github.com/OpenDriveLab/PPGeo.